A Performance Comparison of Open-Source Erasure Coding Libraries for Storage Applications

Erasure coding is a fundemental technique to prevent data loss in storage systems composed of multiple disks. Recently, there have been multiple open-source implementations of a variety of erasure codes. In this work, we present a comparison of the performance of various codes and implementations, concentrating on encoding and decoding. It is hard to draw overarching conclusions from a single performance study. However, performance data is important to gain an understanding of the real-life performance ramifications of code properties and implementation decisions. The significance of this paper is to guide those who use and design codes, so that they may be able to predict what performance to expect when using an erasure code. One important, although obvious, conclusion is that reducing cache misses is more important than reducing XOR operations.